Adjusting Bias in Concept Learning

We approach concept learning as a heuristic search through a space of concepts for a concept that satisfies the learning task at hand The heuristics represent bias that the concept learning program employs when forming an inductive generalization. We present a model of bias adjustment and report our experience with an implementation of the model. A major objective in research on inductive concept learning is creation of a program that can accept training data, apply knowledge, and form inductive hypotheses of the concept, all without human intervention. The learning program searches a space of hypotheses for those that are consistent with the observed examples, and which classify the unobserved instances as indicated by heuristics. The heuristics, which we call bias, determine the inductive generalizations that the program will form, given some set of training examples. The initial bias may be appropriate for one learning task, yet inappropriate for another. In many concept learning programs to date, e.g. (Michalski 83), (Mitchell 77), the search for appropriate bias is done by hand. We examine an approach to mechanizing that search. Vere (Vere 80) and Lenat (Lenat 83) have programs that adjust their bias. Vere uses the set difference operator to construct a new description C = A-B when no other consistent description is available. Lenat's EURISKO learns heuristics that lead the program to discover interesting concepts. EURISKO is an advance from Lenat's AM where the search for appropriate bias was done by hand. II AN APPROACH TO ADJUSTING BIAS We represent bias as a restricted search space of concepts that we call the concept description language. We use Mitchell's Candidate Elimination Algorithm (Mitchell 77) to maintain a version space of all concept descriptions in the concept description language that are consistent with the training instances. When the trainer supplies a positive instance, all concept descriptions that exclude the instance are refuted and removed from the version space. Similarly, when the trainer supplies a negative instance, all concept descriptions that include the instance are refuted and removed from the version space. As shown in Figure 1, as the version space shrinks, the complementary set of refuted hypotheses grows. If the version space becomes empty, then all descriptions in the concept description language have been refuted. A refuted concept description language indicates a refuted bias, and is sufficient justification for adjusting bias. We do not yet know stronger justifications The role of bias adjustment …